Human intelligence emerges from our combination of senses and language abilities. Maybe the same is true for artificial intelligence.

AI systems, on the other hand, are built to do only one of these things at a time. Computer-vision and audio-recognition algorithms can sense things but cannot use language to describe them. A natural- language model can manipulate words, but the words are detached from any sensory reality. If senses and language were combined to give an AI a more human-like way to gather and process new information, could it finally develop something like an understanding of the world? The hope is that these "multimodal" systems, with access to both the sensory and linguistic "modes" of human intelligence, should give rise to a more robust kind of AI that can adapt more easily to new situations or problems.